Refactor of the PropagationModule and removal of WebModule #6033

manuel-alvarez-alvarez · 2023-10-11T19:57:00Z

What Does This Do

Refactoring of the tainting API by moving all tainting operations to the PropagationModule with methods according to:

taint methods are used to taint values : taint(context?, target, source_attributes+)
taintIfTainted methods are used to taint values only if the input is tainted : taintIfTainted(context?, target, input, source_attributes+)
taintIfAnyTainted methods are used to taint values only if any of the inputs is tainted : taintIfAnyTainted(context?, target, input[], source_attributes+)

Motivation

Both modules WebModule and PropagationModule have grown organically bringing a lot of duplication, this PR tries to merge all tainting operations providing a cohesive API.

Additional Notes

This PR is quite big but it does not introduce any new functionality, just a reorganization of the tainting API.

Jira ticket: APPSEC-10454

pr-commenter · 2023-10-11T20:38:28Z

Benchmarks

Startup

Parameters

	Baseline	Candidate
commit	1.23.0-SNAPSHOT~a012761e16	1.23.0-SNAPSHOT~072e68a521
config	baseline	candidate

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
module	Agent	Agent
parent	None	None
variant	iast	iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 54 cases.

Startup time reports for insecure-bank

gantt
    title insecure-bank - global startup overhead: candidate=1.23.0-SNAPSHOT~072e68a521, baseline=1.23.0-SNAPSHOT~a012761e16

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.032 s) : 0, 1031748
Total [baseline] (8.759 s) : 0, 8759255
Agent [candidate] (1.027 s) : 0, 1026874
Total [candidate] (8.764 s) : 0, 8763927
section iast
Agent [baseline] (1.171 s) : 0, 1171310
Total [baseline] (9.349 s) : 0, 9348814
Agent [candidate] (1.155 s) : 0, 1154731
Total [candidate] (9.271 s) : 0, 9270759
section iast_TELEMETRY_OFF
Agent [baseline] (1.143 s) : 0, 1143493
Total [baseline] (9.312 s) : 0, 9312482
Agent [candidate] (1.147 s) : 0, 1147284
Total [candidate] (9.246 s) : 0, 9245937

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.032 s	-
Agent	iast	1.171 s	139.562 ms (13.5%)
Agent	iast_TELEMETRY_OFF	1.143 s	111.745 ms (10.8%)
Total	tracing	8.759 s	-
Total	iast	9.349 s	589.559 ms (6.7%)
Total	iast_TELEMETRY_OFF	9.312 s	553.227 ms (6.3%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.027 s	-
Agent	iast	1.155 s	127.857 ms (12.5%)
Agent	iast_TELEMETRY_OFF	1.147 s	120.41 ms (11.7%)
Total	tracing	8.764 s	-
Total	iast	9.271 s	506.832 ms (5.8%)
Total	iast_TELEMETRY_OFF	9.246 s	482.01 ms (5.5%)

gantt
    title insecure-bank - break down per module: candidate=1.23.0-SNAPSHOT~072e68a521, baseline=1.23.0-SNAPSHOT~a012761e16

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (645.19 ms) : 0, 645190
BytebuddyAgent [candidate] (642.159 ms) : 0, 642159
GlobalTracer [baseline] (295.556 ms) : 0, 295556
GlobalTracer [candidate] (294.505 ms) : 0, 294505
AppSec [baseline] (49.647 ms) : 0, 49647
AppSec [candidate] (48.964 ms) : 0, 48964
Remote Config [baseline] (683.009 µs) : 0, 683
Remote Config [candidate] (677.767 µs) : 0, 678
Telemetry [baseline] (6.146 ms) : 0, 6146
Telemetry [candidate] (6.058 ms) : 0, 6058
section iast
BytebuddyAgent [baseline] (785.25 ms) : 0, 785250
BytebuddyAgent [candidate] (773.117 ms) : 0, 773117
GlobalTracer [baseline] (277.418 ms) : 0, 277418
GlobalTracer [candidate] (274.07 ms) : 0, 274070
AppSec [baseline] (48.892 ms) : 0, 48892
AppSec [candidate] (46.311 ms) : 0, 46311
Remote Config [baseline] (1.247 ms) : 0, 1247
Remote Config [candidate] (578.151 µs) : 0, 578
Telemetry [baseline] (8.238 ms) : 0, 8238
Telemetry [candidate] (11.272 ms) : 0, 11272
IAST [baseline] (15.406 ms) : 0, 15406
IAST [candidate] (14.954 ms) : 0, 14954
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (763.725 ms) : 0, 763725
BytebuddyAgent [candidate] (766.779 ms) : 0, 766779
GlobalTracer [baseline] (273.667 ms) : 0, 273667
GlobalTracer [candidate] (274.258 ms) : 0, 274258
AppSec [baseline] (46.74 ms) : 0, 46740
AppSec [candidate] (46.287 ms) : 0, 46287
Remote Config [baseline] (582.785 µs) : 0, 583
Remote Config [candidate] (607.005 µs) : 0, 607
Telemetry [baseline] (6.796 ms) : 0, 6796
Telemetry [candidate] (9.01 ms) : 0, 9010
IAST [baseline] (17.649 ms) : 0, 17649
IAST [candidate] (15.876 ms) : 0, 15876

Startup time reports for petclinic

gantt
    title petclinic - global startup overhead: candidate=1.23.0-SNAPSHOT~072e68a521, baseline=1.23.0-SNAPSHOT~a012761e16

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.027 s) : 0, 1027447
Total [baseline] (9.348 s) : 0, 9347892
Agent [candidate] (1.033 s) : 0, 1032660
Total [candidate] (9.326 s) : 0, 9326114
section appsec
Agent [baseline] (1.121 s) : 0, 1121300
Total [baseline] (9.389 s) : 0, 9388998
Agent [candidate] (1.114 s) : 0, 1114184
Total [candidate] (9.355 s) : 0, 9354766
section iast
Agent [baseline] (1.159 s) : 0, 1158952
Total [baseline] (9.517 s) : 0, 9516657
Agent [candidate] (1.152 s) : 0, 1151679
Total [candidate] (9.511 s) : 0, 9510660
section profiling
Agent [baseline] (1.209 s) : 0, 1208580
Total [baseline] (9.589 s) : 0, 9588982
Agent [candidate] (1.217 s) : 0, 1216876
Total [candidate] (9.576 s) : 0, 9575790

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.027 s	-
Agent	appsec	1.121 s	93.854 ms (9.1%)
Agent	iast	1.159 s	131.505 ms (12.8%)
Agent	profiling	1.209 s	181.133 ms (17.6%)
Total	tracing	9.348 s	-
Total	appsec	9.389 s	41.107 ms (0.4%)
Total	iast	9.517 s	168.765 ms (1.8%)
Total	profiling	9.589 s	241.09 ms (2.6%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.033 s	-
Agent	appsec	1.114 s	81.524 ms (7.9%)
Agent	iast	1.152 s	119.019 ms (11.5%)
Agent	profiling	1.217 s	184.215 ms (17.8%)
Total	tracing	9.326 s	-
Total	appsec	9.355 s	28.652 ms (0.3%)
Total	iast	9.511 s	184.546 ms (2.0%)
Total	profiling	9.576 s	249.676 ms (2.7%)

gantt
    title petclinic - break down per module: candidate=1.23.0-SNAPSHOT~072e68a521, baseline=1.23.0-SNAPSHOT~a012761e16

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (642.296 ms) : 0, 642296
BytebuddyAgent [candidate] (646.206 ms) : 0, 646206
GlobalTracer [baseline] (294.646 ms) : 0, 294646
GlobalTracer [candidate] (295.76 ms) : 0, 295760
AppSec [baseline] (49.208 ms) : 0, 49208
AppSec [candidate] (49.209 ms) : 0, 49209
Remote Config [baseline] (680.815 µs) : 0, 681
Remote Config [candidate] (697.338 µs) : 0, 697
Telemetry [baseline] (6.105 ms) : 0, 6105
Telemetry [candidate] (6.161 ms) : 0, 6161
section appsec
BytebuddyAgent [baseline] (645.984 ms) : 0, 645984
BytebuddyAgent [candidate] (642.272 ms) : 0, 642272
GlobalTracer [baseline] (295.352 ms) : 0, 295352
GlobalTracer [candidate] (293.272 ms) : 0, 293272
AppSec [baseline] (139.032 ms) : 0, 139032
AppSec [candidate] (138.026 ms) : 0, 138026
Remote Config [baseline] (643.199 µs) : 0, 643
Remote Config [candidate] (638.492 µs) : 0, 638
Telemetry [baseline] (5.751 ms) : 0, 5751
Telemetry [candidate] (5.704 ms) : 0, 5704
section iast
BytebuddyAgent [baseline] (777.439 ms) : 0, 777439
BytebuddyAgent [candidate] (771.787 ms) : 0, 771787
GlobalTracer [baseline] (275.153 ms) : 0, 275153
GlobalTracer [candidate] (273.491 ms) : 0, 273491
AppSec [baseline] (46.694 ms) : 0, 46694
AppSec [candidate] (47.134 ms) : 0, 47134
Remote Config [baseline] (1.244 ms) : 0, 1244
Remote Config [candidate] (579.982 µs) : 0, 580
Telemetry [baseline] (6.946 ms) : 0, 6946
Telemetry [candidate] (7.524 ms) : 0, 7524
IAST [baseline] (16.794 ms) : 0, 16794
IAST [candidate] (16.831 ms) : 0, 16831
section profiling
BytebuddyAgent [baseline] (657.163 ms) : 0, 657163
BytebuddyAgent [candidate] (662.71 ms) : 0, 662710
GlobalTracer [baseline] (359.635 ms) : 0, 359635
GlobalTracer [candidate] (362.227 ms) : 0, 362227
AppSec [baseline] (50.321 ms) : 0, 50321
AppSec [candidate] (49.8 ms) : 0, 49800
Remote Config [baseline] (647.036 µs) : 0, 647
Remote Config [candidate] (659.882 µs) : 0, 660
Telemetry [baseline] (6.042 ms) : 0, 6042
Telemetry [candidate] (6.168 ms) : 0, 6168
ProfilingAgent [baseline] (81.015 ms) : 0, 81015
ProfilingAgent [candidate] (81.204 ms) : 0, 81204
Profiling [baseline] (81.041 ms) : 0, 81041
Profiling [candidate] (81.23 ms) : 0, 81230

Load

Parameters

	Baseline	Candidate
commit	1.23.0-SNAPSHOT~a012761e16	1.23.0-SNAPSHOT~072e68a521
config	baseline	candidate
end_time	2023-10-27T09:17:42	2023-10-27T09:34:04
start_time	2023-10-27T09:17:29	2023-10-27T09:33:51

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
variant	iast	iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 22 cases.

Request duration reports for insecure-bank

gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.23.0-SNAPSHOT~072e68a521, baseline=1.23.0-SNAPSHOT~a012761e16
    dateFormat X
    axisFormat %s
section baseline
no_agent (360.147 µs) : 339, 381
.   : milestone, 360,
iast (471.176 µs) : 450, 492
.   : milestone, 471,
iast_FULL (528.003 µs) : 507, 549
.   : milestone, 528,
iast_INACTIVE (430.327 µs) : 409, 451
.   : milestone, 430,
iast_TELEMETRY_OFF (461.486 µs) : 440, 483
.   : milestone, 461,
tracing (430.152 µs) : 409, 451
.   : milestone, 430,
section candidate
no_agent (363.093 µs) : 342, 384
.   : milestone, 363,
iast (461.19 µs) : 441, 482
.   : milestone, 461,
iast_FULL (526.994 µs) : 506, 548
.   : milestone, 527,
iast_INACTIVE (431.855 µs) : 410, 454
.   : milestone, 432,
iast_TELEMETRY_OFF (463.163 µs) : 442, 485
.   : milestone, 463,
tracing (431.538 µs) : 411, 452
.   : milestone, 432,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	360.147 µs [339.297 µs, 380.996 µs]	-
iast	471.176 µs [449.963 µs, 492.388 µs]	111.029 µs (30.8%)
iast_FULL	528.003 µs [507.274 µs, 548.732 µs]	167.857 µs (46.6%)
iast_INACTIVE	430.327 µs [409.372 µs, 451.282 µs]	70.18 µs (19.5%)
iast_TELEMETRY_OFF	461.486 µs [440.386 µs, 482.587 µs]	101.34 µs (28.1%)
tracing	430.152 µs [408.982 µs, 451.322 µs]	70.006 µs (19.4%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	363.093 µs [341.895 µs, 384.292 µs]	-
iast	461.19 µs [440.545 µs, 481.835 µs]	98.097 µs (27.0%)
iast_FULL	526.994 µs [506.101 µs, 547.886 µs]	163.9 µs (45.1%)
iast_INACTIVE	431.855 µs [410.184 µs, 453.526 µs]	68.762 µs (18.9%)
iast_TELEMETRY_OFF	463.163 µs [441.569 µs, 484.758 µs]	100.07 µs (27.6%)
tracing	431.538 µs [410.945 µs, 452.131 µs]	68.445 µs (18.9%)

Request duration reports for petclinic

gantt
    title petclinic - request duration [CI 0.99] : candidate=1.23.0-SNAPSHOT~072e68a521, baseline=1.23.0-SNAPSHOT~a012761e16
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.348 ms) : 1329, 1367
.   : milestone, 1348,
appsec (1.713 ms) : 1689, 1738
.   : milestone, 1713,
iast (1.448 ms) : 1424, 1472
.   : milestone, 1448,
profiling (1.451 ms) : 1426, 1475
.   : milestone, 1451,
tracing (1.461 ms) : 1436, 1486
.   : milestone, 1461,
section candidate
no_agent (1.369 ms) : 1349, 1388
.   : milestone, 1369,
appsec (1.685 ms) : 1661, 1710
.   : milestone, 1685,
iast (1.457 ms) : 1433, 1481
.   : milestone, 1457,
profiling (1.497 ms) : 1472, 1523
.   : milestone, 1497,
tracing (1.457 ms) : 1432, 1482
.   : milestone, 1457,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.348 ms [1.329 ms, 1.367 ms]	-
appsec	1.713 ms [1.689 ms, 1.738 ms]	365.532 µs (27.1%)
iast	1.448 ms [1.424 ms, 1.472 ms]	100.156 µs (7.4%)
profiling	1.451 ms [1.426 ms, 1.475 ms]	102.938 µs (7.6%)
tracing	1.461 ms [1.436 ms, 1.486 ms]	113.483 µs (8.4%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.369 ms [1.349 ms, 1.388 ms]	-
appsec	1.685 ms [1.661 ms, 1.71 ms]	316.519 µs (23.1%)
iast	1.457 ms [1.433 ms, 1.481 ms]	88.442 µs (6.5%)
profiling	1.497 ms [1.472 ms, 1.523 ms]	128.675 µs (9.4%)
tracing	1.457 ms [1.432 ms, 1.482 ms]	88.262 µs (6.4%)

manuel-alvarez-alvarez added type: refactoring comp: asm iast Application Security Management (IAST) labels Oct 11, 2023

smola approved these changes Oct 16, 2023

View reviewed changes

manuel-alvarez-alvarez mentioned this pull request Oct 16, 2023

Fix IAST ranges for StringBuffer.toString #6050

Closed

manuel-alvarez-alvarez force-pushed the malvarez/iast-taint-api-refactor branch 3 times, most recently from 0efacb4 to 995f5c4 Compare October 22, 2023 20:27

manuel-alvarez-alvarez marked this pull request as ready for review October 22, 2023 21:23

manuel-alvarez-alvarez requested review from a team as code owners October 22, 2023 21:23

manuel-alvarez-alvarez requested review from jandro996, mcculls and dougqh October 22, 2023 21:23

manuel-alvarez-alvarez self-assigned this Oct 23, 2023

manuel-alvarez-alvarez added run-tests: all Run all tests tag: no release notes Changes to exclude from release notes labels Oct 23, 2023

manuel-alvarez-alvarez requested review from smola and DDJavierSantos October 23, 2023 07:45

manuel-alvarez-alvarez force-pushed the malvarez/iast-taint-api-refactor branch from 03bcf09 to 333c7a8 Compare October 23, 2023 09:20

smola approved these changes Oct 23, 2023

View reviewed changes

manuel-alvarez-alvarez force-pushed the malvarez/iast-taint-api-refactor branch from 333c7a8 to 9f8489a Compare October 23, 2023 16:06

jandro996 approved these changes Oct 24, 2023

View reviewed changes

manuel-alvarez-alvarez force-pushed the malvarez/iast-taint-api-refactor branch from 9f8489a to 29e3032 Compare October 24, 2023 09:18

manuel-alvarez-alvarez mentioned this pull request Oct 24, 2023

Improve http.request.path IAST source for servlet #6091

Closed

DDJavierSantos approved these changes Oct 25, 2023

View reviewed changes

manuel-alvarez-alvarez force-pushed the malvarez/iast-taint-api-refactor branch 4 times, most recently from 2116662 to 469769d Compare October 25, 2023 15:24

manuel-alvarez-alvarez force-pushed the malvarez/iast-taint-api-refactor branch from 469769d to e52e4ee Compare October 26, 2023 09:58

smola approved these changes Oct 26, 2023

View reviewed changes

manuel-alvarez-alvarez force-pushed the malvarez/iast-taint-api-refactor branch 2 times, most recently from 547364a to 3a6030d Compare October 27, 2023 08:32

Refactor of the PropagationModule and removal of WebModule

072e68a

manuel-alvarez-alvarez force-pushed the malvarez/iast-taint-api-refactor branch from 3a6030d to 072e68a Compare October 27, 2023 08:50

manuel-alvarez-alvarez merged commit 00a81bb into master Oct 27, 2023
119 checks passed

manuel-alvarez-alvarez deleted the malvarez/iast-taint-api-refactor branch October 27, 2023 10:00

github-actions bot added this to the 1.23.0 milestone Oct 27, 2023

jandro996 pushed a commit that referenced this pull request Oct 27, 2023

Refactor of the PropagationModule and removal of WebModule (#6033)

b68b8f0

jandro996 pushed a commit that referenced this pull request Oct 30, 2023

Refactor of the PropagationModule and removal of WebModule (#6033)

ae66378

manuel-alvarez-alvarez mentioned this pull request Oct 30, 2023

Add name in sources for vertx 4 header names and param values #6114

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Refactor of the PropagationModule and removal of WebModule #6033

Refactor of the PropagationModule and removal of WebModule #6033

manuel-alvarez-alvarez commented Oct 11, 2023 •

edited

Loading

pr-commenter bot commented Oct 11, 2023 •

edited

Loading

Refactor of the PropagationModule and removal of WebModule #6033

Refactor of the PropagationModule and removal of WebModule #6033

Conversation

manuel-alvarez-alvarez commented Oct 11, 2023 • edited Loading

What Does This Do

Motivation

Additional Notes

pr-commenter bot commented Oct 11, 2023 • edited Loading

Benchmarks

Startup

Parameters

Summary

Load

Parameters

Summary

manuel-alvarez-alvarez commented Oct 11, 2023 •

edited

Loading

pr-commenter bot commented Oct 11, 2023 •

edited

Loading